System and method for delivering media based on viewer behavior

ABSTRACT

Methods, systems and devices are provided for communicating multimedia content over a service provider network. A network server may receive an audio-video stream from a content provider server, relaying the audio-video stream to a receiver device via the service provider network, receive an operating mode notification identifying a user behavior from the receiver device, and ceasing the relaying of the audio-video stream to the receiver device based on the user behavior. The network server may also alter the audio-video stream based on the received operating mode notification and send the altered audio-video stream to the receiver device via the service provider network.

BACKGROUND

Wired and wireless communication technologies have improved over thepast several years, and service providers are now able offer their usersa wide array of features and services over high capacity cable, fiber,and/or wireless networks. As a result, the popularity and usage ofInternet protocol (IP) based services (e.g., streaming multimedia, IPtelevision, games, etc.) has grown rapidly, and the improvements innetwork throughput afforded by the high capacity networks is beingmatched by the increase in user usage demands for IP services. By way ofexample, on-demand Internet streaming media services (e.g., Netflix®,Vudu®, Amazon®, etc.) can now account for forty percent of the totalInternet traffic at peak hours. As these IP-based services continue togrow in popularity and usage, the efficient management of the amount ofbandwidth consumed by streaming media services is of increasing economicimportance.

SUMMARY

The various embodiments include methods of communicating multimediacontent over a service provider network, which may include: receiving anaudio-video stream from a content provider server, relaying theaudio-video stream to a receiver device via the service providernetwork, receiving an operating mode notification from the receiverdevice, and ceasing the relaying of the audio-video stream based on auser behavior identified by the received operating mode notification. Inan embodiment, the method may include altering the audio-video streambased on the received operating mode notification, and sending thealtered audio-video stream to the receiver device via the serviceprovider network. In a further embodiment, altering the audio-videostream based on the received operating mode notification may includegenerating an audio stream based on the audio-video stream.

In a further embodiment, the method may include detecting the userbehavior in a device processor of the receiver device, selecting anoperating mode in the device processor based on the detected userbehavior, generating the operating mode notification in the deviceprocessor based on the selected operating mode, and sending thegenerated operating mode notification to the network server processorvia the service provider network.

In a further embodiment, detecting the user behavior may includedetecting that a user is not actively viewing content rendered on anelectronic display of the receiver device. In a further embodiment,detecting that the user is not actively viewing content rendered on theelectronic display of the receiver device may include activating acamera of the receiver device. In a further embodiment, detecting thatthe user is not actively viewing content rendered on the electronicdisplay of the receiver device may include determining whether the userhas scrolled off a video player portion of a web browser.

Further embodiments may include a computing device having a processorconfigured with processor-executable instructions to perform variousoperations corresponding to the methods discussed above.

Further embodiments may include a computing device having various meansfor performing functions corresponding to the various method operationsdiscussed above.

Further embodiments may include a non-transitory processor-readablestorage medium having stored thereon processor-executable instructionsconfigured to cause a processor to perform various operationscorresponding to the method operations discussed above.

Further embodiments may include a communication system that includes aserver configured to transmit data via a service provider network and areceiver device configured to receive data via the service providernetwork. The server and receiver device may include processorsconfigured with processor-executable instructions to perform variousoperations corresponding to the methods discussed above.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and constitutepart of this specification, illustrate exemplary embodiments of theinvention, and together with the general description given above and thedetailed description given below, serve to explain the features of theinvention.

FIG. 1 is a communication system block diagram illustrating networkcomponents of a multimedia communication system suitable for use invarious embodiments.

FIG. 2 is a block diagram illustrating sample components of a receiverdevice that may be configured to detect user behavior in accordance withvarious embodiments.

FIG. 3 is a block diagram illustrating example network components andinformation flows in an embodiment communication system.

FIG. 4 is a block diagram illustrating example functional components ofa network server configured to alter a multimedia stream in accordancewith an embodiment.

FIG. 5 is a process flow diagram illustrating an embodiment networkserver method of streaming multimedia content to a receiver device basedon detected user behavior.

FIG. 6 is a process flow diagram illustrating an embodiment networkserver method of streaming multimedia content to a receiver device basedon detected user behavior.

FIG. 7 is a process flow diagram illustrating an embodiment networkserver method of streaming multimedia content to a receiver device basedon the availability of network resources and detected user behavior.

FIG. 8 is a process flow diagram illustrating an embodiment receiverdevice method of receiving multimedia content based on detected userbehavior.

FIG. 9 is a system block diagram of a receiver device suitable for usewith any of the embodiments.

FIG. 10 is a system block of a server suitable for use with any of theembodiments.

DETAILED DESCRIPTION

The various embodiments are described in detail with reference to theaccompanying drawings. Wherever possible, the same reference numbers areused throughout the drawings to refer to the same or like parts.References made to particular examples and implementations are forillustrative purposes, and are not intended to limit the scope of theinvention or the claims.

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any implementation described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other implementations.

The term “mobile device” is used herein to refer to any or all ofsmartphones, personal or mobile multi-media players, personal dataassistants (PDA's), laptop computers, tablet computers, ultrabooks,media players, multimedia Internet enabled cellular telephones, wirelessgaming controllers, and similar personal electronic devices whichinclude a programmable processor and circuitry for receiving andprocessing multimedia information.

The term “receiver device” is used herein to refer to set-top-boxes,televisions, computing devices, mobile devices, and similar electronicdevices that include circuitry for receiving multimedia information.

The term “service provider network” is used generically herein to referto any network suitable for providing consumers with access to theInternet or IP services over broadband connections, and may encompassboth wired and wireless networks/technologies. Examples of wired networktechnologies and networks that may be included within a service providernetwork include cable networks, fiber optic networks, hybrid-fiber-cablenetworks, networks that implement the data over cable service interfacespecification (DOCSIS), networks that utilize asymmetric digitalsubscriber line (ADSL) technologies, etc. Examples of wireless networktechnologies and networks that may be included within a service providernetwork include third generation partnership project (3GPP), long termevolution (LTE) systems, third generation wireless mobile communicationtechnology (3G), fourth generation wireless mobile communicationtechnology (4G), global system for mobile communications (GSM),universal mobile telecommunications system (UMTS), high-speed downlinkpacket access (HSDPA), 3GSM, general packet radio service (GPRS), codedivision multiple access (CDMA) systems (e.g., cdmaOne, CDMA2000™),enhanced data rates for GSM evolution (EDGE), advanced mobile phonesystem (AMPS), digital AMPS (IS-136/TDMA), evolution-data optimized(EV-DO), digital enhanced cordless telecommunications (DECT), WorldwideInteroperability for Microwave Access (WiMAX), wireless local areanetwork (WLAN), Wi-Fi Protected Access I & II (WPA, WPA2), Bluetooth®,land mobile radio (LMR), and integrated digital enhanced network (iden).Each of these wired and wireless technologies involves, for example, thetransmission and reception of data, signaling and/or content messages.

Any references to terminology and/or technical details related to anindividual wired or wireless communications standard or technology arefor illustrative purposes only, and not intended to limit the scope ofthe claims to a particular communication system or technology unlessspecifically recited in the claim language.

As mentioned above, IP and streaming multimedia services (e.g.,Netflix®, Vudu®, Amazon®, YouTube®, etc.) are growing in popularity anduse. For example, Netflix®, a subscription-based movie and televisionshow rental service that streams video content over the Internet,currently has over twenty-six million subscribers and is one of thelargest sources of web traffic in North America. Similarly, YouTube®(which is a video-sharing website that streams music videos, movies, TVshows, etc.) continues to grow in popularity and use, and now accountsfor a significant portion of the total web traffic in North America. Dueto the growth in popularity and usage of these and other IP andstreaming media services, multimedia and video data now consume asignificant portion of a service provider network's available resources(e.g., bandwidth, etc.).

To meet customer demands and manage increases in network traffic,service providers and multiple system operators (MSOs) are beginning toshift their subscription plans from a flat-rate unlimited usage model toan actual usage model in which subscribers are charged usage fees basedon the amount of bandwidth and services they consume per billing period.In addition, many content providers and MSOs are changing their contentdelivery models from a more traditional video broadcast model to anIP-based video delivery model (e.g., unicast, etc.). Due to the largeamount of information typically included in these video streams, videodata is expected to become the primary source of web traffic andcongestion in the future, and subscribers are expected to be chargedsubstantial usage fees for viewing videos online.

The various embodiments provide systems, devices, and methods ofefficiently communicating multimedia content (e.g., audio-video streams,etc.) over a service provider network to reduce the amount of networkresources (e.g., bandwidth, etc.) consumed by IP and streamingmultimedia services. Various embodiments enable a network server toselectively stream a subset of the content (e.g., multimedia data)received from a content sever to a receiver device (e.g., set-top-boxes,televisions, computing devices, mobile phones, etc.) based on detecteduser behavior and/or the availability of device or network resources.Various embodiments intelligently and dynamically adjust or elect mediatransport protocols based on the detected user behavior. Variousembodiments reduce the amount of information communicated to a receiverdevice over the service provider network, thereby reducing web traffic,network congestion, and subscriber usage fees.

In addition to consuming network resources (e.g., bandwidth, etc.),video data may consume a significant portion of a receiver device'savailable processing and power resources. For example, displaying videocontent on a portable receiver device (e.g., smartphone, etc.) generallyrequires the execution of power and resource intensive softwareapplications (e.g., media players) and/or multimedia processing tasks,which may greatly reduce the battery life and/or otherwise deplete thereceiver device's resources. By way of example, a typical smartphoneprocessor may consume 10-20 milliamps (mA) of current when idle, andaround 400-600 mA of current when executing a multimedia task. This isdue, in part, to the large amount of digital information that must bereceived, error-corrected, decoded, and processed by a receiver devicein order to properly display the received video data.

The various embodiments reduce the amount of digital informationreceived, error-corrected, decoded, and processed by a receiver deviceby enabling a receiver device to receive a subset of the content (e.g.,multimedia data) streamed/downloaded from a content provider sever basedon user behavior and/or the availability of device resources. Variousembodiments also enable a receiver device to adjust its resource usage(e.g., turn off its display, enter a low power mode, cease videodecoding operations, etc.) based on the received content and/or thedetected user behavior or actions.

There are currently a number of readily available audio and videocompression techniques (e.g., moving picture experts group “MPEG”compression, etc.) that reduce the spatial and temporal redundancy ofthe video to reduce the amount of information that is streamed/sentacross the network. However, regardless of the efficiency of thecompression methods, multimedia and video data still consume asignificant amount of network resources (e.g., bandwidth, etc.) due tothe enormous amount of encoded information that must be transmitted overthe network. In addition, receiver devices may be required to performcomplex and power intensive video processing and decoding operations todecode compressed video files into a format suitable for rendering thevideo content on an electronic display. These decoding operations mayconsume a significant amount of the receiver devices' power, processing,and memory resources. The various embodiments reduce the amount ofinformation transmitted over the network, and eliminate or reduce thecomplexity of the processing operations performed by a receiver devicein presenting content to users.

In addition to the compression techniques discussed above, existingsolutions may employ adaptive bitrate streaming (ABS) techniques toreduce the amount of network resources consumed by video and multimediadata. These ABS technologies generally adjust the quality of a videostream so that the bitrate of the video stream is commensurate with thereceiver device's capabilities. By way of example, dynamic adaptivestreaming over HTTP (DASH) is an ABS technology that adjusts the qualityof video streams based on the receiving device's available bandwidth andCPU capabilities. However, DASH and other ABS technologies only operateto reduce the quality of the audio-video content included in the streamreceived by the receiver device. For example, DASH and other ABS systemstypically include a video encoder server that encodes a single rawaudio-video file at multiple bit rates to generate the multiple versionsof the same audio-video file. Each of the multiple versions of theaudio-video file includes the same audio and video content, but atvarying qualities (e.g., high definition, standard definition, 1080p,720i, etc.). A video player client may request to receive one version ofthe multiple available versions based on the availability of resources.In this manner, the video player client may balance presentation qualityagainst device resource availability by electing the most suitable videostream for reception. However, regardless of which stream the videoplayer client elects to receive, the video player client always receivesthe same audio and video content.

As discussed above, existing ABS solutions simply adjust video qualityby selecting one of a plurality of video streams based on the bandwidthand the CPU capabilities of the receiving device. In contrast, thevarious embodiments dynamically adjust the content of a multimedia oraudio-video stream (i.e., as opposed to simply adjusting the bit-rate ofthe received stream) based on detected user behaviors or actions (i.e.,as opposed to bandwidth or processing capabilities of a receiverdevice). For example, various embodiments may extract audio informationfrom an existing audio-video stream and stream only the audioinformation to the receiver device based on determining that there is ahigh probability that the user is not actively viewing the video contentand/or is only interested in listening to the audio content. As anotherexample, various embodiments may terminate or delay the transmission ofthe entire audio-video stream based on determining that the user is notin close proximity to the receiver device.

The various embodiments may be implemented within a variety ofmulti-media delivery systems and/or communication networks, includingbroadcast networks, multicast networks, unicast networks, wirednetworks, wireless networks, cable television networks, over-the-airtelevision broadcast networks, and/or any communication system in whichaudio and visual content may be aggregated and selected for transmissionto the end user.

FIG. 1 illustrates example components and communication links in acommunication network 100 suitable for implementing the variousembodiments. Receiver devices 102 (e.g., smart-phones, laptops,set-top-boxes, televisions, etc.) may receive multimedia content viacommunication links (wired or wireless) to one or more network serversand systems 126, which may provide a connection to the Internet 108and/or content servers 130. The wired communication links may beachieved via cable, fiber, or any medium suitable for communicating IPdata or audio-visual content. Wireless communication links may be viashort-range radio technologies (e.g., Bluetooth®, WiFi, etc.), viapeer-to-peer connections, or via any other known wireless communicationtechnology.

Receiver devices 102 may also receive multimedia content via a unicastnetwork 106, such as a cellular network, WiFi network, WiMAX, etc. Atypical unicast network 106 includes a plurality of unicast transmitters118 coupled to a controller 120 and/or network operations center (NOC),which operate to coordinate the delivery of content to receiver devices102. To facilitate IP and streaming data services (e.g., streamingaudio-video feeds), the unicast network 106 may include one or morenetwork servers and systems 124 that provide a connection to theInternet 108. Receiver devices 102 may communicate with the unicastnetwork 106 via two-way wireless communication links 122 (e.g., toreceive multimedia content streams from the Internet 108 and/or thecontent servers 130, etc.).

FIG. 2 illustrates sample components of a receiver device 102 that maybe configured to detect user behavior and elect to receive a subset of amultimedia stream in accordance with various embodiments. Specifically,FIG. 2 illustrates that the receiver device 102 may include a processor202 coupled to internal memory 204, an electronic display 206, amicrophone 208, a camera 210, sensors 226, and/or to a speaker 212. Thereceiver device 102 may further include user interface 220 elements(e.g., buttons, icons, etc.) for receiving user inputs, which may behard key buttons, soft key buttons, touch keys, or any input meanssuitable for receiving a user input.

The sensors 226 may include one or more sensors configured to detectmotion, gestures (e.g., hand movements), eye movements, and/or otheruser actions, which when recognized by the receiver device 102, maycause the receiver device 102 to execute a specific command oroperation. The sensors 226 may also include one or more sensors forscanning/collecting information from the user's environment (e.g., room,etc.), geo-spatial positioning sensors (e.g., GPS transceiver, etc.),distance measuring sensors (e.g., a laser, sonic range finder, etc.),orientation sensors (e.g., up, down, level, etc.), etc. The sensors 226may further include an eye tracking component configured to detect eyemovements or the presence or the location of the user's eye (e.g., viathe camera 210) relative to the receiver device 102 or electronicdisplay 206.

The receiver device 102 may include communications circuitry 214 coupledto the processor 202 for establishing data connections with a network,such as a local area network coupled to servers or computers of aservice provider network. Additionally, the receiver device 102 mayinclude a transceiver 216, a wireless radio 222 (e.g., Bluetooth® radio,WiFi radio, etc.), and/or an antenna 218 for sending and receivingelectromagnetic radiation and/or connecting to a wireless data link.

The processor 202 and memory 204 may be configured to receiveaudio-video content from a network server (e.g., a content server, anapplication server, etc.) and process and present the receivedaudio-video content on the electronic display 206 and speaker 212. Thereceiver device 102 may communicate with the network server via a directcommunication link (e.g., wireless data link, etc.), through a centralserver, via short-range radio technologies (e.g., Bluetooth®, WiFi,etc.), via peer-to-peer connections, or via any other knowncommunication technologies.

In an embodiment, the receiver device 102 may be configured to detectuser behavior, determine an operating mode for the receiver device 102based on the detected user behavior, set a current operating mode of thereceiver device 102 to the determined operating mode, and adjust thereceiver device's 102 usage of resources based on the operating mode.For example, the receiver device 102 may be configured to detect when auser is not actively viewing the video content being rendered on theelectronic display 206, determine that the operating mode should be setto “audio only,” and set the operating mode to “audio only,” andpower-off the electronic display 206. Similarly, the receiver device 102may determine that a detected user behavior is indicative of the user'sintention to view video content, determine that the operating modeshould be set to “full mode,” set the operating mode to “full,” turn onthe electronic display 206, and render video content on the electronicdisplay 206. As a further example, the receiver device 102 may beconfigured to determine that a detected user behavior is indicative ofthe user's absence from the vicinity of the receiver device 102,determine that the operating mode should be set to “power-off,” andde-energize or disconnect the electronic display 206, speaker 212,processor 202, etc. from a power source.

As discussed above, the receiver device 102 may be configured to detectuser behaviors. In various embodiments, detecting user behaviors mayinclude determining that the user is not viewing or paying attention tothe electronic display 206 (e.g., via information collected from acamera, motion sensor, etc.), determining that the user is not in closeproximity to the receiver device 102, and/or determining that the useris engaged in an activity (e.g., driving, entering text in a differentbrowser tab, etc.) that indicates there is a relatively high probabilitythat the user is not actively viewing and/or listening to content.

In an embodiment, detecting user behaviors may include the receiverdevice 102 receiving user input, such as an input indicating that theuser only desires to consume the audio portions of an audio-video file,is currently driving, etc. In an embodiment, detecting user behaviorsmay include accessing information stored in memory, such asconfiguration files, settings, preferences, etc.

In an embodiment, detecting user behaviors may include gatheringinformation from software applications (e.g., calendar application,browser, etc.), geo-spatial positioning and navigation systems (e.g.,global positioning system or “GPS”), and/or other sensors of thereceiver device 102 (e.g., microphone 208, camera 210, motion sensor,speaker 212, etc.).

In an embodiment, detecting user behaviors may include analyzing useractions and/or interactions with the receiver device 102, such asdetermining whether the user is typing, texting, making a voice call,moving, opening a new browser tab, minimizing the browser tab/pagedisplaying the audio-video feed, scrolling past the video player in awebpage, viewing an electronic program guide, etc.

As discussed above, the receiver device 102 may be configured todetermine an operating mode based on the detected user behavior andadjust its usage of resources based on the operating mode. In variousembodiments, the operating modes may include a full mode, an audio-onlymode, a video-only mode, a local mode, a closed-caption mode, a sleepmode, a low-power mode, and/or a power-off mode. By way of example, thereceiver device 102 may be configured to present both audio and videocontent when in full mode, present only audio content when in audio-onlymode, and present only video content when in video-only mode. Adjustingthe usage of resources may include powering on or off the receiverdevice 102 or any combination of the receiver device's components. Forexample, adjusting the usage of resources may include powering on both adisplay and a speaker when in full mode, powering off a display when inaudio-only mode, powering off a speaker when in video-only mode, andpowering off both a display and a speaker when in any of the sleep, lowpower or power-off modes.

In various embodiments, the operating modes may further includetime-delayed versions of the audio-only, video-only, local,closed-caption, sleep, low power, and/or power-off modes. For example,the receiver device 102 may be configured to present both audio andvideo content for a predetermined amount of time after determining thatthe display is to be powered off (i.e., before entering the audio-onlymode) when in the time-delayed audio-only mode. Likewise, the receiverdevice 102 may perform similar or corresponding operations for each ofthe time-delayed versions of the video-only, local, sleep, low power,and power-off modes.

In an embodiment, the receiver device 102 may be configured to terminateaudio and/or video feeds (e.g., turn off wireless radios, etc.) andswitch to presenting audio and/or video content stored on memory whenentering the local mode.

In an embodiment, the receiver device 102 may be configured to power offthe electronic display 206, and activate a text-to-speech softwaremodule to convert closed caption information into audio when in theclosed-captioned mode.

In an embodiment, the receiver device 102 may be configured to determinean operating mode for the receiver device 102 based on the availabilityof network or receiver device resources (e.g., bandwidth, availablesystem memory, processor utilization, remaining battery power, etc.).

FIG. 3 illustrates example components and information flows in acommunication system 300 suitable for implementing various embodiments.The communication system 300 may include a content provider server 302,a network server 304, and a receiver device 102. The content providerserver 302 may send an audio-video file/stream 308 to the network server304 for relaying to the receiver device 102 via the service providernetwork 306.

In various embodiments, the network server 304 may be included in (or aspart of) a service provider network 306, in a local area network (LAN),in a wide area network (WAN), the Internet, etc. The network server 304may be coupled to a service provider network 306 (e.g., via theInternet, etc.) and/or the receiver device 102 (e.g., via a LAN, etc.).In an embodiment, the network server 304 may be a proxy server.

As discussed above, the receiver device 102 may be configured to detectuser behaviors and determine an operating mode based on the detecteduser behaviors. In an embodiment, the receiver device 102 may be furtherconfigured to notify the network server 304 of the detected userbehaviors and/or operating mode. For example, the receiver device 102may notify the network server 304 each time the receiver device 102changes its current operating mode to a new or different operating mode.

The network server 304 may be configured to receive the operating modenotification from the receiver device 102, and terminate or suspendsending the audio-video file/stream 310 based on the operating mode(e.g., local mode, power-off mode, etc.). The network server 304 mayalso be configured to modify, alter, or transform the audio-videofile/stream 308 based on the operating mode of the receiver device 102,and send the modified/altered/transformed audio-video file/stream 310 tothe receiver device 102 (i.e., in lieu of the original audio-videofile/stream 308 received from the content provider server 302). Forexample, the network server 304 may receive a notification from thereceiver device 102 indicating that the operating mode should be set to“audio only,” transform an audio-video file to an audio file, and sendthe audio file to the receiver device 102 in lieu of the audio-videofile. Since the audio file will contain significantly less informationthan the audio-video file, sending the audio file in lieu of theaudio-video file will reduce the amount of information communicated overthe service provider network 306.

In an embodiment, generating the modified audio-video file/stream 310may include generating an enhanced audio stream by upsampling,denoising, or otherwise enhancing (e.g., converting standard definitionaudio to a high definition audio, etc.) the audio content. For example,the network server 304 may separate the audio content from the videocontent, discard the video content, enhance the audio content, and sendthe enhanced audio stream to the receiver device 102 as the modifiedaudio-video file/stream 310. In this manner, the various embodiments mayenhance the user experience while lowering network resourcesconsumption.

In an embodiment, the receiver device 102 may be configured to generateand store a timestamp when it enters a new operating mode and/or detectsa relevant user behavior. The timestamp may be communicated to thenetwork server 304 to indicate a time (or position, frame, etc.) afterwhich the user stopped consuming audio and/or video content or a time(or position, frame, etc.) after which the receiver device 102 stoppedpresenting audio and/or video content to the user (e.g., via theelectronic display/speakers). The network server 304 may storeaudio-video content based on the received timestamp(s) for latertransmission to the receiver device 102 (e.g., when the user returns toconsuming the audio and/or video content). In this manner, a user mayautomatically return to consuming audio and/or video content from wherehe/she left off. Moreover, the user may choose to return to either theposition/frame where the receiver device 102 determined that the userstopped consuming the audio/video or the position/frame in which thereceiver device 102 stopped presenting the audio/video.

In an embodiment, the receiver device 102 and/or network server 304 maybe configured to monitor the amount of network resources (e.g.,bandwidth, etc.) saved by terminating, delaying, altering and/ormodifying audio-video streams 310 over a given time period (e.g.,billing period). In this manner, the receiver device 102 and/or networkserver 304 may track how much money the user saved in each billingperiod as a result of the reductions in network resource consumption.Similarly, the various embodiments may monitor reductions in networktraffic and resource usage resulting from terminating, delaying,altering and/or modifying audio-video streams 310.

As discussed above, the network server 304 may be configured to alterthe audio-video stream 310 based on the operating mode, and send thealtered audio-video file/stream 310 to the receiver device 102. In anembodiment, altering the audio-video stream 310 may include selecting astream (e.g., audio-only stream, etc.) from a plurality of availablestreams (e.g., audio-video stream, video-only stream, audio-only stream,etc.). In other embodiments, altering the audio-video stream 310 mayinclude converting a file from a first format (e.g., MPEG, etc.) into asecond format (MP3, etc.), de-multiplexing a multiplexed audio-videostream, and/or selecting or altering a transmission or session setupprotocol (e.g., via session initiation protocol, MPEG transport stream,etc.).

FIG. 4 illustrates example functional components of an exemplary networkserver 304 configured to alter a multimedia or audio-video stream inaccordance with an embodiment. In the example illustrated in FIG. 4, thenetwork server 304 includes a receiver module 402, a controller module404, an encoder module 406, a memory 408, and a transmitter module 412.

The receiver module 402 may be configured to receive communicationsignals containing a multiplex of information (including audio and/orvideo streams, multimedia and/or data packets, frames, etc.), andprocess the received communication signals (e.g., demodulate, performmultiprotocol-encapsulation forward error correction, etc.) to extractthe multiplexed information.

The encoder module 406 may be configured to decode/process themultiplexed information to generate multiple, independent streams,files, or data units (e.g., based on a manifest file, etc.). Forexample, the encoder module 404 may decode/process an audio-video streamto generate audio data and video data, encode the audio data in a firststream, and encode the video data into a second stream.

The controller module 404 may be configured to determine which of theplurality of generated streams are to be sent to the receiver devicebased on the operating mode and/or select one or more streams fortransmission to the receiver device (i.e., via the transmission module410).

In various embodiments, the encoder 406 may also be configured to encodethe audio and/or video data at multiple bit rates and/or into aplurality of streams. In these embodiments, the controller 404 maygenerate a manifest file that includes information pertaining theavailable streams (e.g., differing bit rates, communication channelsetc.), which may be sent to the receiver device in advance.

FIG. 5 illustrates an embodiment network server method 500 of streamingmultimedia content to a receiver device based on detected user behavior.In block 502, a network server may begin receiving an audio-video streamfrom a content provider server. In block 504, the network server maybegin relaying the received audio-video stream to a receiver device(e.g., over a service provider network). In block 506, the networkserver may receive an operating mode notification identifying a useraction, a change in user behavior, and/or an operating mode of thereceiver device (e.g., full mode, an audio-only mode, a video-only mode,etc.). For example, the network server may receive an operating modenotification indicating that the user has scrolled off the media playerportion of a web browser, has had his/her eyes closed for an extendedperiod of time or is otherwise not actively viewing the content, thereceiver device is entering the audio-only mode, etc.

In block 508, the network server may begin storing the audio-videostream from the content provider server in a memory. In block 510, thenetwork server may alter the audio-video stream from the contentprovider server based on the received notification. For example, thenetwork server may convert the audio-video stream into an audio-onlystream. In block 512, the network server may stop relaying theaudio-video stream to the receiver device. In block 514, the networkserver may begin sending the altered stream (e.g., audio-video stream)to the receiver device. In block 516, the network server may receiveanother operating mode notification from the receiver device. Forexample, the network server may receive an operating mode notificationindicating that the receiver device is entering the full mode. In block518, the network server may resume sending the audio-video stream to thereceiver device.

In an embodiment, in block 518, the network server may send the storedaudio-video stream to the receiver device. In this embodiment, thenetwork server may continue to receive and store the audio-video streamfrom the content provider server while retrieving and sending the storedaudio-video stream to the receiver device.

FIG. 6 illustrates another embodiment network server method 600 ofstreaming multimedia content to a receiver device based on detected userbehavior/actions. In block 602, a network server may begin receiving anaudio-video stream from a content provider server. In block 604, thenetwork server may begin relaying the received audio-video stream to areceiver device. In block 606, the network server may receive anoperating mode notification indicating that the user of the receiverdevice is no longer in close proximity to the receiver device. In block608, the network server may begin storing the audio-video stream fromthe content provider server in memory. In block 610, the network servermay stop relaying the audio-video stream to the receiver device. Inblock 612, the network server may receive an operating mode notificationfrom the receiver device indicating that the user is now in closeproximity to the receiver device. In block 614, the network server maysend the stored audio-video stream to the receiver device.

FIG. 7 illustrates an embodiment network server method 700 of streamingmultimedia content to a receiver device based on detected userbehavior/actions and the availability of network resources. In block702, a network server may begin receiving an audio-video stream from acontent provider server. In block 704, the network server may beginrelaying the received audio-video stream to a receiver device. In block706, the network server may receive an operating mode notificationindicating that the user is not actively viewing the content. In block708, the network server may evaluate, analyze or determine thedegree/level of network congestion and/or the availability of networkresources.

In determination block 710, the network server may determine whether thenetwork is congested. When the network server determines that thenetwork is congested (i.e., determination block 710=“Yes”), in block712, the network server may generate a closed caption data stream basedon the audio-video stream received from a content provider server. Whenthe network server determines that the network is not congested (i.e.,determination block 710=“No”), in block 714, the network server maygenerate an audio-only stream based on the audio-video stream receivedfrom a content provider server. In block 716, the network server maystop relaying the audio-video stream to the receiver device. In block718, the network server may send the generated audio-video stream to thereceiver device.

FIG. 8 illustrates an embodiment receiver device method 800 of reducingthe consumption of network and device resources based on detected userbehavior. In block 802, a receiver device may begin receiving anaudio-video stream from a network server. In block 804, the receiverdevice may present audio and video content to the user (e.g., via anelectronic display, speakers, etc.). In block 806, the receiver devicemay monitor the user actions and/or behaviors. For example, the receiverdevice may determine whether the display is powered on, whether the userhas scrolled off the media player portion of a browser, periodicallyactivate a camera to determine if the user's eyes are visibly open,activate an eye tracking component to determine if the user's eyemovements are consistent with active viewing of the presented content,activate motion sensors to determine if the user is in close proximityto the receiver device, activate orientation sensors to determine if thereceiver device is face down, etc.

In block 808, the receiver device may detect a change in userbehavior/action (e.g., user is no longer actively viewing the content,etc.). In block 810, the receiver device may compute, determine, orselect an operating mode based on the detected user behavior/actionand/or the availability of resources. For example, the receiver devicemay select an “audio-only” mode based on determining that the user is nolonger actively viewing the content.

In block 812, the receiver device may generate a operating modenotification message and send the operating mode notification to thenetwork server. In block 814, the receiver device may modify the usageof resources on the receiver device (e.g., powering off a display,de-energizing a video processor, etc.).

In block 816, the receiver device may begin receiving an altered stream(e.g., an audio-only stream, etc.) from the network server. In block820, the receiver device may present content based on theselected/current operating mode and or the content included in thealtered stream. For example, the receiver device may present only audiocontent (e.g., via the speakers) based on entering an audio-only modeand/or based on receiving an audio-only stream. The operations of block806-820 may be repeated until the stream is terminated, the receiverdevice is powered off, the user elects to interrupt the operations, etc.

FIG. 9 is a system block diagram of a mobile receiver device in the formof a laptop computer suitable for use with any of the embodiments. Atypical laptop computer 900 may include a processor 901 coupled to anelectronic display 904, a speaker 906, a volatile memory 902 and to anonvolatile memory, such as a disk drive 922. The laptop computer 900may also include a floppy disc drive, compact disc (CD) or DVD discdrive coupled to the processor 901. The laptop computer 900 may includean antenna 910, a multimedia receiver 912, a transceiver 918 and/orcommunications circuitry 920 coupled to the processor 901 for sendingand receiving electromagnetic radiation, connecting to a wireless datalink, and receiving IP and streaming multimedia services (e.g.,audio-video streams, information flows, etc.). Additionally, the laptopcomputer 900 may include network access ports 924 coupled to theprocessor 901 for establishing data connections with a network (e.g.,LAN coupled to a service provider network, etc.). A laptop computer 900typically also includes a keyboard 914 and a mouse pad 916 for receivinguser inputs.

The various network components, severs, and systems may be implementedon any of a variety of commercially available server devices, such asthe server 1000 illustrated in FIG. 10. Such a server 1000 typicallyincludes a processor 1001 coupled to volatile memory 1002 and a largecapacity nonvolatile memory, such as a disk drive 1003. The server 1000may also include a floppy disc drive, compact disc (CD) or DVD discdrive 1004 coupled to the processor 1001. The server 1000 may alsoinclude network access ports 1006 coupled to the processor 1001 forestablishing data connections with a network 1005, such as a local areanetwork coupled to other computers, servers, or components in a serviceprovider network.

The processors 901, 1001 may be any programmable microprocessor,microcomputer or multiple processor chip or chips that can be configuredby software instructions (applications) to perform a variety offunctions, including the functions of the various embodiments describedabove. In some mobile receiver devices, multiple processors may beprovided, such as one processor dedicated to wireless communicationfunctions, one processor dedicated to video processing, and oneprocessor dedicated to running other applications. Typically, softwareapplications may be stored in the internal memory 902, 1002 before theyare accessed and loaded into the processor 901, 1001. The processor 901,1001 may include internal memory sufficient to store the applicationsoftware instructions.

The foregoing method descriptions and the process flow diagrams areprovided merely as illustrative examples and are not intended to requireor imply that the steps of the various embodiments must be performed inthe order presented. As will be appreciated by one of skill in the artthe steps in the foregoing embodiments may be performed in any order.Words such as “then,” “next,” etc. are not intended to limit the orderof the steps; these words are simply used to guide the reader throughthe description of the methods. Although process flow diagrams maydescribe the operations as a sequential process, many of the operationscan be performed in parallel or concurrently. In addition, the order ofthe operations may be re-arranged. A process may correspond to a method,a function, a procedure, a subroutine, a subprogram, etc. When a processcorresponds to a function, its termination may correspond to a return ofthe function to the calling function or the main function.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as hardware, software, or any combinationthereof. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, circuits,and steps have been described above generally in terms of theirfunctionality. Whether such functionality is implemented as hardware orsoftware depends upon the particular application and design constraintsimposed on the overall system. Skilled artisans may implement thedescribed functionality in varying ways for each particular application,but such implementation decisions should not be interpreted as causing adeparture from the scope of the present invention.

The hardware used to implement the various illustrative logics, logicalblocks, modules, and circuits described in connection with theembodiments disclosed herein may be implemented or performed with ageneral purpose processor, a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA) or other programmable logic device, discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. Ageneral-purpose processor may be a microprocessor, but, in thealternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration. Alternatively, some blocks or methods may be performed bycircuitry that is specific to a given function.

Embodiments implemented in computer software may be implemented insoftware, firmware, middleware, microcode, hardware descriptionlanguages, or any combination thereof. A code segment ormachine-executable instructions may represent a procedure, a function, asubprogram, a program, a process, a task, a tread, a routine, asubroutine, a module, a software package, a class, or any combination ofinstructions, data structures, or program statements. A code segment maybe coupled to another code segment or a hardware circuit by passingand/or receiving information, data, arguments, parameters, or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted via any suitable means including memorysharing, message passing, token passing, network transmission, etc.

When implemented in software, the functions may be stored as one or moreinstructions or code on a non-transitory computer-readable orprocessor-readable storage medium. The steps of a method or algorithmdisclosed herein may be embodied in a processor-executable softwaremodule which may reside on a computer-readable or processor-readablestorage medium. Non-transitory computer-readable or processor-readablemedia includes both computer storage media and tangible storage mediathat facilitate transfer of a computer program from one place toanother. Non-transitory processor-readable storage media may be anyavailable media that may be accessed by a computer. By way of example,and not limitation, such non-transitory processor-readable media maycomprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othertangible storage medium that may be used to store desired program codein the form of instructions or data structures and that may be accessedby a computer or processor. Disk and disc, as used herein, includecompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above should also be included within the scope ofcomputer-readable media. Additionally, the operations of a method oralgorithm may reside as one or any combination or set of codes and/orinstructions on a non-transitory processor-readable medium and/orcomputer-readable medium, which may be incorporated into a computerprogram product.

When implemented in hardware, the functionality may be implementedwithin circuitry of a signal processing circuit that may be suitable foruse in a wireless receiver or mobile device. Such a wireless signalprocessing circuit may include circuits for accomplishing the signalmeasuring and calculating steps described in the various embodiments.

Any reference to claim elements in the singular, for example, using thearticles “a,” “an” or “the,” is not to be construed as limiting theelement to the singular.

The preceding description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the following claims and theprinciples and novel features disclosed herein.

1.-25. (canceled)
 26. A method of communicating multimedia content overa service provider network, the method comprising: receiving, by areceiver device processor in a receiver device, an audio-video streamfrom a network server; commencing presentation, by the receiver deviceprocessor, at least one of audio content or video content; monitoring,by the receiver device processor, user behaviors during the presentationof the least one of audio content or video content to detect changes inuser actions or behaviors; determining, by the receiver deviceprocessor, an operating mode based on the detected user behaviors;generating and sending, by the receiver device processor, an operatingmode notification message to the network server; receiving, by thereceiver device processor, an altered stream from the network server inresponse to sending the operating mode notification message to thenetwork server; and rendering, by the receiver device processor, contentbased on the content included in the altered stream.
 27. The method ofclaim 26, further comprising modifying resource usage on the receiverdevice in response to generating and sending an operating modenotification message to the network server.
 28. The method of claim 27,wherein modifying resource usage on the receiver device comprisesde-energizing a video processor of the receiver device or powering offan electronic display of the receiver device.
 29. The method of claim26, wherein receiving the altered stream from the network server inresponse to sending the operating mode notification message to thenetwork server comprises receiving an audio-only stream from the networkserver in response to sending the operating mode notification message tothe network server.
 30. The method of claim 26, wherein receiving thealtered stream from the network server in response to sending theoperating mode notification message to the network server comprisesreceiving a closed-caption-only data stream from the network server inresponse to sending the operating mode notification message to thenetwork server.
 31. The method of claim 26, wherein: monitoring the userbehaviors during the presentation of the least one of audio content orvideo content to detect the changes in user actions or behaviorscomprises monitoring the user behaviors during the presentation of theleast one of audio content or video content to determine whether a userof the receiver device is actively viewing the presented content; anddetermining the operating mode based on the detected user behaviorscomprises determining the operating mode in response to detecting thatthe user is no longer activity viewing the presented content.
 32. Themethod of claim 31, wherein monitoring the user behaviors during thepresentation of the least one of audio content or video content todetermine whether the user is actively viewing the content comprises:activating a camera of the receiver device to determine whether theuser's eyes are visibly open; activating an eye tracking component ofthe receiver device to determine whether the user's eye movements areconsistent with active viewing of the presented content; activating amotion sensor to determine whether the user is in close proximity to thereceiver device; or activating orientation sensors to determine whetherthe receiver device is face down.
 33. The method of claim 31, whereinmonitoring the user behaviors during the presentation of the least oneof audio content or video content to determine whether the user isactively viewing the content comprises monitoring a sensor in thereceiver device that detects one or more of: a motion; a gesture; anorientation; an eye movement; a presence of the user's eye; or alocation of the user's eye relative to the receiver device.
 34. Areceiver device, comprising: a receiver device processor configured withprocessor-executable software instructions to: receive an audio-videostream from a network server; commence presenting at least one of audiocontent or video content; monitor user behaviors during the presentationof the least one of audio content or video content to detect changes inuser actions or behaviors; determine an operating mode based on thedetected user behaviors; generate and send an operating modenotification message to the network server; receive an altered streamfrom the network server in response to sending the operating modenotification message to the network server; and render content based onthe content included in the altered stream.
 35. The receiver device ofclaim 34, wherein the receiver device processor is further configured tomodify resource usage on the receiver device in response to generatingand sending an operating mode notification message to the networkserver.
 36. The receiver device of claim 35, wherein the receiver deviceprocessor is configured to modify resource usage on the receiver deviceby de-energizing a video processor of the receiver device or poweringoff an electronic display of the receiver device.
 37. The receiverdevice of claim 34, wherein the receiver device processor is configuredto receive the altered stream from the network server in response tosending the operating mode notification message to the network server byreceiving an audio-only stream from the network server in response tosending the operating mode notification message to the network server.38. The receiver device of claim 34, wherein the receiver deviceprocessor is configured to receive the altered stream from the networkserver in response to sending the operating mode notification message tothe network server by receiving a closed-caption-only data stream fromthe network server in response to sending the operating modenotification message to the network server.
 39. The receiver device ofclaim 34, wherein the receiver device processor is configured to:monitor the user behaviors during the presentation of the least one ofaudio content or video content to detect the changes in user actions orbehaviors by monitoring the user behaviors during the presentation ofthe least one of audio content or video content to determine whether auser of the receiver device is actively viewing the presented content;and determine the operating mode based on the detected user behaviors bydetermining the operating mode in response to detecting that the user isno longer activity viewing the presented content.
 40. The receiverdevice of claim 39, wherein the receiver device processor is configuredto monitor the user behaviors during the presentation of the least oneof audio content or video content to determine whether the user isactively viewing the content by: activating a camera of the receiverdevice to determine whether the user's eyes are visibly open; activatingan eye tracking component of the receiver device to determine whetherthe user's eye movements are consistent with active viewing of thepresented content; activating a motion sensor to determine whether theuser is in close proximity to the receiver device; or activatingorientation sensors to determine whether the receiver device is facedown.
 41. The receiver device of claim 39, wherein the receiver deviceprocessor is configured to monitor the user behaviors during thepresentation of the least one of audio content or video content todetermine whether the user is actively viewing the content by monitoringa sensor in the receiver device that detects one or more of: a motion; agesture; an orientation; an eye movement; a presence of the user's eye;or a location of the user's eye relative to the receiver device.
 42. Anon-transitory computer readable storage medium having stored thereonprocessor-executable software instructions configured to cause areceiver device processor to perform operations for communicatingmultimedia content over a service provider network, the operationscomprising: receiving an audio-video stream from a network server;commencing presentation at least one of audio content or video content;monitoring user behaviors during the presentation of the least one ofaudio content or video content to detect changes in user actions orbehaviors; determining an operating mode based on the detected userbehaviors; generating and sending an operating mode notification messageto the network server; receiving an altered stream from the networkserver in response to sending the operating mode notification message tothe network server; and rendering content based on the content includedin the altered stream.
 43. The non-transitory computer readable storagemedium of claim 42, wherein the stored processor-executable softwareinstructions configured to cause the receiver device processor toperform operations further comprising modifying resource usage on thereceiver device in response to generating and sending an operating modenotification message to the network server.
 44. The non-transitorycomputer readable storage medium of claim 43, wherein the storedprocessor-executable software instructions configured to cause thereceiver device processor to perform operations such that modifyingresource usage on the receiver device comprises de-energizing a videoprocessor of the receiver device or powering off an electronic displayof the receiver device.
 45. The non-transitory computer readable storagemedium of claim 42, wherein the stored processor-executable softwareinstructions configured to cause the receiver device processor toperform operations such that receiving the altered stream from thenetwork server in response to sending the operating mode notificationmessage to the network server comprises receiving an audio-only streamfrom the network server in response to sending the operating modenotification message to the network server.
 46. The non-transitorycomputer readable storage medium of claim 42, wherein the storedprocessor-executable software instructions configured to cause thereceiver device processor to perform operations such that receiving thealtered stream from the network server in response to sending theoperating mode notification message to the network server comprisesreceiving a closed-caption-only data stream from the network server inresponse to sending the operating mode notification message to thenetwork server.
 47. The non-transitory computer readable storage mediumof claim 42, wherein the stored processor-executable softwareinstructions configured to cause the receiver device processor toperform operations such that: monitoring the user behaviors during thepresentation of the least one of audio content or video content todetect the changes in user actions or behaviors comprises monitoring theuser behaviors during the presentation of the least one of audio contentor video content to determine whether a user of the receiver device isactively viewing the presented content; and determining the operatingmode based on the detected user behaviors comprises determining theoperating mode in response to detecting that the user is no longeractivity viewing the presented content.
 48. The non-transitory computerreadable storage medium of claim 47, wherein the storedprocessor-executable software instructions configured to cause thereceiver device processor to perform operations such that monitoring theuser behaviors during the presentation of the least one of audio contentor video content to determine whether the user is actively viewing thecontent comprises: activating a camera of the receiver device todetermine whether the user's eyes are visibly open; activating an eyetracking component of the receiver device to determine whether theuser's eye movements are consistent with active viewing of the presentedcontent; activating a motion sensor to determine whether the user is inclose proximity to the receiver device; or activating orientationsensors to determine whether the receiver device is face down.
 49. Thenon-transitory computer readable storage medium of claim 47, wherein thestored processor-executable software instructions configured to causethe receiver device processor to perform operations such that monitoringthe user behaviors during the presentation of the least one of audiocontent or video content to determine whether the user is activelyviewing the content comprises monitoring a sensor in the receiver devicethat detects one or more of: a motion; a gesture; an orientation; an eyemovement; a presence of the user's eye; or a location of the user's eyerelative to the receiver device.